Coordinated Resource Provisioning in Federated Grids

A fundamental problem in building large scale Grid resource sharing system is the need for efficient and scalable techniques for discovery and provisioning of resources for delivering expected Quality of Service (QoS) to users’ applications. The current approaches to Grid resource sharing based on resource brokers are non-coordinated since these brokers make scheduling related decisions independent of the others in the system. Clearly, this worsens the load-sharing and utilisation problems of distributed Grid resources as sub-optimal schedules are likely to occur. Further, existing brokering systems rely on centralised information services for resource discovery. Centralised or hierarchical resource discovery systems are prone to single-point failure, lack scalability and fault-tolerance ability. In the centralised model, the network links leading to the server are very critical to the overall functionality of the system, as their failure might halt the entire distributed system operation. In this thesis, I propose a new federated Grid system, called Grid-Federation that aims towards decentralised and coordinated coupling of distributed Grid resources as a part of single cooperative system. The Peer-to-Peer (P2P) network model forms the basis for the design of decentralised protocols for scheduling and resource discovery in the GridFederation. The decentralised organisation enhances the scalability and reliability of the system. Two levels of decentralised coordination is presented in this thesis: (i) a Service Level Agreement (SLA) based broker-to-broker coordination protocol that inhibits the brokers from over-provisioning the resources and also gives every site the admission control capability; and (ii) a P2P tuple space based coordination protocol that coordinates the scheduling process among the distributed resource brokers. The thesis demonstrates the effectiveness of the proposed Grid-Federation based decentralised protocols in coordinating scalable and robust resource management through extensive simulation studies. Thesis Contributions: To support the thesis that Grid-Federation model along with its decentralised protocols for scheduling and coordination is the best possible way to implement new generation Grid resource sharing system I have: • outlined key taxonomies for decentralised Grid resource sharing systems, • proposed, modeled, and evaluated a decentralised resource sharing system called Grid-Federation, which aims toward policy based cooperative and coordinated coupling of distributed cluster resources, • proposed, modeled, and evaluated an SLA based broker-to-broker service contract negotiation protocol, • proposed, modeled, and evaluated the extension of Distributed Hash Tables (DHTs) such as Chord, Pastry using a spatial publish/subscribe index to support decentralised resource discovery protocols in the Grid-Federation, • proposed, modeled, and evaluated a DHT-based tuple space for coordinating the application schedules among distributed brokers, • designed and implemented the Alchemi-Federation software system, that supports a coordinated federation of Alchemi cluster over a DHT-based resource discovery system.